VS-CAM: Vertex Semantic Class Activation Mapping to Interpret Vision Graph Neural Network

Graph convolutional neural network (GCN) has drawn increasing attention and attained good performance in various computer vision tasks, however, there is a lack of clear interpretation GCN’s inner mechanism. For standard networks (CNNs), class activation mapping (CAM) methods are commonly used to visualize the connection between CNN’s decision image region by generating heatmap. Nonetheless, such heatmap usually exhibits semantic-chaos when these CAMs applied GCN directly. In this paper, we proposed novel visualization method particularly applicable GCN, Vertex Semantic Class Activation Mapping (VS-CAM). VS-CAM includes two independent pipelines produce set semantic-probe maps semantic-base map, respectively. Semantic-probe detect semantic information from map aggregate semantic-aware Qualitative results show that can obtain heatmaps where highlighted regions match objects much more precisely than CNN-based CAM. The quantitative evaluation further demonstrates superiority VS-CAM.